Methods and apparatus for application prediction through machine learning based analysis of IO patterns

ABSTRACT

An apparatus comprises a host device configured to communicate over a network with a storage system. The host device comprises a multi-path input-output driver configured to control delivery of input-output operations from the host device to the storage system over selected ones of a plurality of paths through the network. The multi-path input-output driver is further configured to collect information characterizing input-output patterns for each of a plurality of different applications executing on the host device, and to provide the collected information to a machine learning system. The machine learning system utilizes the collected information to generate one or more application predictions based at least in part on analysis of observed input-output patterns. The collected information in some embodiments comprises statistical distributions of numbers of input-output operations having payload sizes falling into specified ranges over a given time period.

FIELD

The field relates generally to information processing systems, and moreparticularly to storage in information processing systems.

BACKGROUND

Storage arrays and other types of storage systems are often shared bymultiple host devices over a network. However, it can be difficult inthese and other situations to satisfy the particular input-output (IO)processing performance requirements of the potentially large numbers ofdistinct applications that may execute at different times on one or morehost devices. As a result, the overall IO processing performance acrossthe multiple distinct applications can be substantially less thanoptimal under certain conditions. Accordingly, a need exists forimproved techniques for processing IO operations from multiple distinctapplications sharing a storage system over a network.

SUMMARY

Illustrative embodiments provide techniques for application predictionthrough analysis of TO patterns using machine learning. Such techniquescan be implemented in a host device and/or a storage system.

For example, in some host device embodiments, a multi-path layer of oneor more host devices is configured to include functionality forapplication prediction through machine learning based analysis of IOpatterns in conjunction with communication with a storage array or othertype of storage system via a storage area network (SAN) or other type ofnetwork. The multi-path layer illustratively comprises at least onemulti-path input-output (MPIO) driver configured to process TOoperations of at least one host device. Some storage system embodimentscan be similarly configured to include functionality for applicationprediction through machine learning based analysis of IO patterns.

These and other arrangements can allow a host device and/or a storagesystem to better adapt limited IO processing resources as a function ofapplication predictions generated by a machine learning system, therebyproviding better overall IO processing performance across multipledistinct applications.

In one embodiment, an apparatus comprises a host device configured tocommunicate over a network with a storage system. The host devicecomprises an MPIO driver configured to control delivery of IO operationsfrom the host device to the storage system over selected ones of aplurality of paths through the network.

The MPIO driver is further configured to collect informationcharacterizing IO patterns for each of a plurality of differentapplications executing on the host device, and to provide the collectedinformation to a machine learning system. The machine learning systemutilizes the collected information to generate one or more applicationpredictions based at least in part on analysis of observed IO patterns.

The collected information in some embodiments comprises statisticaldistributions of numbers of IO operations having payload sizes fallinginto specified ranges over a given time period. Such embodiments utilizethese payload size statistics to learn characteristics of the typical IOpatterns of particular applications over time so as to be able torecognize, identify or otherwise predict those applications fromobserved IO patterns. Additional or alternative statistics or othertypes of collected information can be used in other embodiments.

The machine learning system is illustratively implemented in the hostdevice, in the storage system, or partially in the host device andpartially in the storage system. For example, in some embodiments, themachine learning system is implemented within the MPIO driver, althoughnumerous other arrangements are possible.

At least one of the host device and the storage system is illustrativelyconfigured to utilize the one or more application predictions generatedby the machine learning system to adjust one or more parameters relatedto its processing of IO operations for the corresponding applications.For example, in some embodiments at least one of the host device and thestorage system may be configured to allocate a relatively high level ofprocessing resources to IO operations associated with a firstapplication prediction, and to allocate a relatively low level ofprocessing resources to IO operations associated with a secondapplication prediction different than the first application prediction.

These and other illustrative embodiments include, without limitation,apparatus, systems, methods and computer program products comprisingprocessor-readable storage media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an information processing system configuredwith functionality for application prediction through machine learningbased analysis of IO patterns in an illustrative embodiment.

FIG. 2 is a flow diagram of a process for application prediction throughmachine learning based analysis of IO patterns in an illustrativeembodiment.

FIG. 3 is a block diagram showing multiple layers of a layered systemarchitecture that includes functionality for application predictionthrough machine learning based analysis of IO patterns in anillustrative embodiment.

FIGS. 4A, 4B and 4C, collectively referred to herein as FIG. 4 , showdifferent aspects of an example implementation of application predictionthrough machine learning based analysis of IO patterns in anillustrative embodiment.

DETAILED DESCRIPTION

Illustrative embodiments will be described herein with reference toexemplary information processing systems and associated computers,servers, storage devices and other processing devices. It is to beappreciated, however, that embodiments of the present disclosure are notrestricted to use with the particular illustrative system and deviceconfigurations shown. Accordingly, the term “information processingsystem” as used herein is intended to be broadly construed, so as toencompass, for example, processing systems comprising cloud computingand storage systems, as well as other types of processing systemscomprising various combinations of physical and virtual processingresources. An information processing system may therefore comprise, forexample, at least one data center that includes one or more cloudshosting multiple tenants that share cloud resources. Numerous othertypes of enterprise and cloud-based computing and storage systems arealso encompassed by the term “information processing system” as thatterm is broadly used herein.

FIG. 1 shows an information processing system 100 configured inaccordance with an illustrative embodiment. The information processingsystem 100 comprises a plurality of host devices 102-1, 102-2, . . .102-N. The host devices 102 communicate over a storage area network(SAN) 104 with at least one storage array 105. The storage array 105comprises a plurality of storage devices 106-1, . . . 106-M each storingdata utilized by one or more applications running on one or more of thehost devices 102. The storage devices 106 are illustratively arranged inone or more storage pools. The storage array 105 and its associatedstorage devices 106 are an example of what is more generally referred toherein as a “storage system.” This storage system in the presentembodiment is shared by the host devices 102, and is therefore alsoreferred to herein as a “shared storage system.” Other embodiments caninclude only a single host device, possibly configured to have exclusiveuse of the storage system.

The host devices 102 illustratively comprise respective computers,servers or other types of processing devices capable of communicatingwith the storage array 105 of the SAN 104. For example, at least asubset of the host devices 102 may be implemented as respective virtualmachines of a compute services platform or other type of processingplatform. The host devices 102 in such an arrangement illustrativelyprovide compute services such as execution of one or more applicationson behalf of each of one or more users associated with respective onesof the host devices 102.

The term “user” herein is intended to be broadly construed so as toencompass numerous arrangements of human, hardware, software or firmwareentities, as well as combinations of such entities.

Compute and/or storage services may be provided for users under aPlatform-as-a-Service (PaaS) model, an Infrastructure-as-a-Service(IaaS) model and/or a Function-as-a-Service (FaaS) model, although it isto be appreciated that numerous other cloud infrastructure arrangementscould be used. Also, illustrative embodiments can be implemented outsideof the cloud infrastructure context, as in the case of a stand-alonecomputing and storage system implemented within a given enterprise.

The storage devices 106 of the storage array 105 of SAN 104 implementlogical units (LUNs) configured to store objects for users associatedwith the host devices 102. These objects can comprise files, blocks orother types of objects. The host devices 102 interact with the storagearray 105 utilizing read and write commands as well as other types ofcommands that are transmitted over the SAN 104. Such commands in someembodiments more particularly comprise Small Computer System Interface(SCSI) commands, although other types of commands can be used in otherembodiments. A given IO operation as that term is broadly used hereinillustratively comprises one or more such commands. References herein toterms such as “input-output” and “IO” should be understood to refer toinput and/or output. Thus, an IO operation relates to at least one ofinput and output.

Also, the term “storage device” as used herein is intended to be broadlyconstrued, so as to encompass, for example, a logical storage devicesuch as a LUN or other logical storage volume. A logical storage devicecan be defined in the storage array 105 to include different portions ofone or more physical storage devices. Storage devices 106 may thereforebe viewed as comprising respective LUNs or other logical storagevolumes.

Each of the host devices 102 illustratively has multiple paths to thestorage array 105, with at least one of the storage devices 106 of thestorage array 105 being visible to that host device on a given one ofthe paths. A given one of the storage devices 106 may be accessible tothe given host device over multiple paths.

Different ones of the storage devices 106 of the storage array 105illustratively exhibit different latencies in processing of IOoperations. In some cases, the same storage device may exhibit differentlatencies for different ones of multiple paths over which that storagedevice can be accessed from a given one of the host devices 102.

The host devices 102, SAN 104 and storage array 105 in the FIG. 1embodiment are assumed to be implemented using at least one processingplatform each comprising one or more processing devices each having aprocessor coupled to a memory. Such processing devices canillustratively include particular arrangements of compute, storage andnetwork resources. For example, processing devices in some embodimentsare implemented at least in part utilizing virtual resources such asvirtual machines (VMs) or Linux containers (LXCs), or combinations ofboth as in an arrangement in which Docker containers or other types ofLXCs are configured to run on VMs.

The host devices 102 and the storage array 105 may be implemented onrespective distinct processing platforms, although numerous otherarrangements are possible. For example, in some embodiments at leastportions of the host devices 102 and the storage array 105 areimplemented on the same processing platform. The storage array 105 cantherefore be implemented at least in part within at least one processingplatform that implements at least a subset of the host devices 102.

The SAN 104 may be implemented using multiple networks of differenttypes to interconnect storage system components. For example, the SAN104 may comprise a portion of a global computer network such as theInternet, although other types of networks can be part of the SAN 104,including a wide area network (WAN), a local area network (LAN), asatellite network, a telephone or cable network, a cellular network, awireless network such as a WiFi or WiMAX network, or various portions orcombinations of these and other types of networks. The SAN 104 in someembodiments therefore comprises combinations of multiple different typesof networks each comprising processing devices configured to communicateusing Internet Protocol (IP) or other related communication protocols.

As a more particular example, some embodiments may utilize one or morehigh-speed local networks in which associated processing devicescommunicate with one another utilizing Peripheral Component Interconnectexpress (PCIe) cards of those devices, and networking protocols such asInfiniBand, Gigabit Ethernet or Fibre Channel. Numerous alternativenetworking arrangements are possible in a given embodiment, as will beappreciated by those skilled in the art.

The host devices 102 comprise respective sets of IO queues 110-1, 110-2,. . . 110-N and respective MPIO drivers 112-1, 112-2, . . . 112-N. TheMPIO drivers 112 collectively comprise a multi-path layer of the hostdevices 102. The multi-path layer provides functionality for applicationprediction through machine learning based analysis of IO patterns usingrespective instances of path selection logic 114-1, 114-2, . . . 114-Nimplemented within the MPIO drivers 112.

The MPIO drivers 112 may comprise, for example, otherwise conventionalMPIO drivers, such as PowerPath® drivers from Dell EMC, suitablymodified in the manner disclosed herein to implement functionality forapplication prediction. Other types of MPIO drivers from other drivervendors may be suitably modified to incorporate functionality forapplication prediction through machine learning based analysis of IOpatterns as disclosed herein.

The MPIO driver 112-1 is configured to deliver IO operations selectedfrom its corresponding set of IO queues 110-1 to the storage array 105via selected ones of multiple paths over the SAN 104. The sources of theIO operations stored in the set of IO queues 110-1 illustrativelyinclude respective processes of one or more applications executing onthe host device 102-1. Other types of sources of IO operations may bepresent in a given implementation of system 100.

The paths from the host device 102-1 to the storage array 105illustratively comprise paths associated with respectiveinitiator-target pairs, with each initiator comprising a host busadaptor (HBA) or other initiating entity of the host device 102-1 andeach target comprising a port or other targeted entity corresponding toone or more of the storage devices 106 of the storage array 105. Asnoted above, the storage devices 106 illustratively comprise LUNs orother types of logical storage devices.

In some embodiments, the paths are associated with respectivecommunication links between the host device 102-1 and the storage array105 with each such communication link having a negotiated link speed.For example, in conjunction with registration of a given HBA to a switchof the SAN 104, the HBA and the switch may negotiate a link speed. Theactual link speed that can be achieved in practice in some cases is lessthan the negotiated link speed, which is a theoretical maximum value.

Various scheduling algorithms, load balancing algorithms and/or othertypes of algorithms can be utilized by the MPIO driver 112-1 indelivering IO operations from the IO queues 110-1 to the storage array105 over particular paths via the SAN 104. Each such IO operation isassumed to comprise one or more commands for instructing the storagearray 105 to perform particular types of storage-related functions suchas reading data from or writing data to particular logical volumes ofthe storage array 105. Such commands are assumed to have various payloadsizes associated therewith, and the payload associated with a givencommand is referred to herein as its “command payload.”

A command directed by the host device 102-1 to the storage array 105 isconsidered an “outstanding” command until such time as its execution iscompleted in the viewpoint of the host device 102-1, at which time it isconsidered a “completed” command. The commands illustratively compriserespective SCSI commands, although other command formats can be used inother embodiments. A given such command is illustratively defined by acorresponding command descriptor block (CDB) or similar formatconstruct. The given command can have multiple blocks of payloadassociated therewith, such as a particular number of 512-byte SCSIblocks or other types of blocks.

In illustrative embodiments to be described below, it is assumed withoutlimitation that the initiators of a plurality of initiator-target pairscomprise respective HBAs of the host device 102-1 and that the targetsof the plurality of initiator-target pairs comprise respective ports ofthe storage array 105. Examples of such HBAs and storage array ports areillustrated in conjunction with the embodiment of FIG. 3 .

The MPIO driver 112-1 is further configured to collect informationcharacterizing IO patterns for each of a plurality of differentapplications executing on the host device 102-1, and to provide thecollected information to a machine learning system. The machine learningsystem utilizes the collected information to generate one or moreapplication predictions based at least in part on analysis of observedIO patterns.

Additional examples of arrangements of this type are described below inconjunction with the embodiments of FIGS. 2, 3 and 4 . For example, theoperation of an example machine learning system in generating anapplication prediction based at least in part on analysis of observed IOpatterns is illustrated in detail in FIG. 4 .

At least one of the MPIO driver 112-1, the host device 102-1 and thestorage array 105 is illustratively configured to initiate one or moreautomated actions responsive to a given application prediction generateby the machine learning system.

Such a machine learning system illustratively comprises, for example, adeep learning neural network or other type of neural network, or anothertype of artificial intelligence based system, configured to learn IOpatterns of respective applications over time so as to able to recognizeor otherwise predict, from a given observed IO pattern, the applicationthat generated the IO pattern.

In some embodiments, the machine learning system more particularlycomprises one or more machine learning based application predictors,such as machine learning based application predictor 120 implemented inthe storage array 105. Additionally or alternatively, one or moremachine learning based application predictors can be implemented in eachof one or more of the host devices 102.

A machine learning system can therefore be implemented at least in partin the storage array 105 and/or one or more of the host devices 102. Forexample, the machine learning system can be implemented at least in partin the MPIO driver 112-1 of the host device 102-1. Numerous alternativearrangements of application predictors and other machine learning systemcomponents can be used in other embodiments.

A given “application prediction” as that term is broadly used herein isintended to encompass, for example, a recognition or otheridentification of a particular application or application type by amachine learning system from observation of an IO pattern having certaincharacteristics that through machine learning the system has associatedwith the particular application or application type.

It is therefore to be appreciated that an application prediction canrecognize or otherwise identify a particular type of application fromamong a plurality of different application types utilizing an observedIO pattern.

Although operations such as collecting information characterizing IOpatterns and providing the collected information to a machine learningsystem are controlled by or otherwise implemented at least in part bythe MPIO driver 112-1 in the present embodiment, this is by way ofexample rather than limitation. In other embodiments, at least portionsof these operations can be carried out elsewhere in the host device102-1.

In some embodiments, collecting information characterizing IO patternsfor each of a plurality of different applications executing on the hostdevice 102-1 comprises collecting information for use in development ofa machine learning model for deployment in the machine learning system.The collected information for use in development of the machine learningmodel illustratively comprises one or more of a training set fortraining the machine learning model to recognize particular types ofapplications from their respective IO patterns, and a testing set forevaluating performance of the machine learning model in recognizingparticular types of applications from their respective IO patterns.

Such training and testing can be done in a separate platform of system100, apart from the host devices 102 and the storage array 105. Forexample, the training and testing can be done on a separate cloud-basedplatform.

Once the model training and testing is complete, the model weights andother related information are stored in one or more files for subsequentdeployment with one or more components of the system 100 to recognize,identify or otherwise predict applications from their respective uniqueIO patterns.

Accordingly, collecting information characterizing IO patterns for eachof a plurality of different applications executing on the host device102-1 illustratively comprises collecting information for use by adeployed machine learning model of the machine learning system ingenerating the one or more application predictions.

In some embodiments, collecting information characterizing IO patternsfor each of a plurality of different applications executing on the hostdevice 102-1 illustratively comprises determining statisticaldistributions of numbers of IO operations having payload sizes fallingwithin specified ranges over a given time period. For example, a set ofsuch payload size ranges is illustratively given by {<=4K, 4K to 64K,64K to 128K, >128K}, although other payload size ranges can be used.

These and other embodiments recognize that particular application typescan have unique distributions of IO operations of different payloadsizes, transaction sequences and other measurable characteristics overtime. These distributions can serve as “fingerprints” of respectivedistinct application types that are recognizable by a trained machinelearning model. Such statistics and other information are collected fromdifferent application types, such as Oracle, Exchange, Sybase, datawarehousing and other applications running on one or more hosts. Therelevant data in some embodiments is extracted using one or more scriptsand saved as individual comma-separated value (CSV) files.

A wide variety of other types of statistics characterizing IO patternscan be collected by the MPIO driver 112-1 and provided to the machinelearning system.

In some embodiments, statistics and other information characterizing IOpatterns are collected by the MPIO driver 112-1 using the PowerPath®Path Management Insight (PMI) feature. For example, PowerPath® PMI canserve as a collection point for statistics or other informationcharacterizing IO patterns, possibly based on counts of read requests,write requests or other types of IO operations having payload sizes thatfall within particular specified ranges of payload sizes within a givenperiod of time.

Additionally or alternatively, weights and other information associatedwith a trained and tested machine learning model can be deployeddirectly into the MPIO driver 112-1, with PowerPath® PMI or anothersimilar feature of the MPIO driver 112-1 being reconfigured to feed IOstatistics directly to the deployed model. The resulting applicationpredictions are illustratively displayed to a user as part of a PMIdisplay output, in addition to or in place of being used to trigger oneor more automated actions.

In some embodiments, the MPIO driver 112-1 is further configured toreceive the one or more application predictions generated by the machinelearning system, and to utilize the one or more application predictionsreceived from the machine learning system to modify a path selectionalgorithm implemented by path selection logic 114-1 of the MPIO driverfor selecting paths for delivery of IO operations from the host device102-1 to the storage array 105 through the SAN 104. Accordingly, thepath selection logic 114-1 can alter its path selection algorithm so asto better accommodate the particular IO processing requirements ofparticular detected application types.

Similarly, the application predictions can be sent to the storage array105 for use in adjusting allocation of its IO processing resources inaccordance with particular detected application types.

As indicated above, a given application prediction generated by themachine learning system illustratively comprises information identifyinga particular one of a plurality of possible application types. At leastone of the host device 102-1 and the storage array 105 is configured toutilize the one or more application predictions generated by the machinelearning system to adjust one or more parameters related to itsprocessing of IO operations for the corresponding applications. Forexample, at least one of the host device 102-1 and the storage array 105is illustratively configured to allocate a relatively high level ofprocessing resources to IO operations associated with a firstapplication prediction, and to allocate a relatively low level ofprocessing resources to IO operations associated with a secondapplication prediction different than the first application prediction.Numerous other resource allocation arrangements may be adjusted,modified or otherwise controlled using the application predictionsgenerated by the machine learning system.

As indicated above, the machine learning system in some embodiments runsat least in part on the storage array 105 and/or the host device 102-1,but can additionally or alternatively run on one or more other ones ofthe host devices 102. For example, it is possible in some embodimentsthat a particular one of the host devices 102 can be dedicated toimplementing a machine learning system for use by other ones of the hostdevices 102.

Responsive to one or more of application predictions generated in themanner described herein, various automated actions can be taken. Forexample, an assignment of one or more initiators to one or more targetscan be adjusted, possibly by shifting at least one of the initiatorsfrom a current target corresponding to a first port of the storage array105 having a relatively high TO processing load to an updated targetcorresponding to a second port of the storage array 105 having arelatively low IO processing load. Additionally or alternatively,various adjustments can be made in at least one mapping of initiators,targets and logical devices that indicates for each of the initiators atleast one target that is to be used to communicate with a given logicaldevice of the storage array 105.

The MPIO driver 112-1 is further configured in some embodiments toinitiate an automated reconfiguration process to redefine zoning andmasking information that characterizes relationships between theinitiators and the targets responsive to one or more applicationpredictions, and/or to initiate an automated path discovery process todiscover new paths through the SAN 104 responsive to one or moreapplication predictions. A wide variety of other automated actions canbe taken based at least in part on application predictions generated bya machine learning system in illustrative embodiments.

These and other operations referred to herein as being performed by orunder the control of the MPIO driver 112-1 can in some embodiments beperformed at least in part outside of the MPIO driver 112-1, such as inthe host device 102-1 generally.

In some embodiments, a predetermined command is sent from the hostdevice 102-1 to the storage array 105, illustratively utilizing the MPIOdriver 112-1, to determine if zoning and masking information has beenchanged. The predetermined command can comprise, for example, a logsense command, a mode sense command, a “vendor unique command” or VUcommand, or combinations of multiple instances of these or othercommands, in an otherwise standardized command format.

As noted above, the MPIO driver 112-1 illustratively initiates one ormore automated actions responsive to an application prediction. Suchautomated actions can also be performed at least in part by the MPIOdriver 112-1, rather than just initiated by the MPIO driver 112-1, butcan additionally or alternatively be performed at least in part by othercomponents of the host device 102-1 or the system 100.

For example, as indicated above, a given automated action initiated bythe MPIO driver 112-1 responsive to an application predictionillustratively comprises automatically adjusting an assignment of one ormore of the initiators to one or more of the targets based at least inpart on the application prediction.

As another example, a given automated action initiated by the MPIOdriver 112-1 responsive to an application prediction illustrativelycomprises initiating automated deployment of one or more additionalpaths associated with respective spare communication links between thehost device 102-1 and the storage array 105.

As a further example, a given automated action initiated by the MPIOdriver 112-1 responsive to the application prediction comprisesinitiating generation of a notification of the application predictionfor delivery to a host administrator.

Additionally or alternatively, an automated action initiated by the MPIOdriver 112-1 responsive to the application prediction can compriseinitiating generation of a notification for delivery to the storagearray 105. Such a notification for delivery to the storage array 105illustratively comprises a “vendor unique command” or VU command in anotherwise standardized command format, such as a SCSI command format,although other command formats may be used.

Yet another example of an automated action that may be initiated by theMPIO driver 112-1 comprises initiating an adjustment to an algorithmutilized in path selection for delivery of IO operations from the set ofIO queues 110-1 to the storage array 105. Such an algorithmillustratively comprises a scheduling algorithm, load balancingalgorithm or other type of algorithm utilized in selecting IO operationsfor delivery over particular selected ones of multiple available paths.A given such algorithm can incorporate both scheduling and loadbalancing functionality.

Selecting a particular one of multiple available paths for delivery of aselected one of the IO operations of the set of IO queues 110-1 is moregenerally referred to herein as “path selection.” Path selection as thatterm is broadly used herein can in some cases involve both selection ofa particular IO operation and selection of one of multiple possiblepaths for accessing a corresponding logical device of the storage array105. The corresponding logical device illustratively comprises a LUN orother logical storage volume to which the particular IO operation isdirected.

As indicated previously, absent use of the techniques for applicationprediction through machine learning based analysis of IO patterns asdisclosed herein, it can be difficult to satisfy the particular IOprocessing performance requirements of the potentially large numbers ofdistinct applications that may execute at different times on the hostdevices 102, possibly leading to deficient performance across themultiple distinct applications.

These and other drawbacks are advantageously overcome in illustrativeembodiments herein by utilization of a multi-path layer comprising oneor more of the MPIO drivers 112 to facilitate application predictionthrough machine learning based analysis of IO patterns as describedabove.

For example, illustrative embodiments disclosed herein advantageouslyallow one or more of the host devices 102 and/or the storage array 105of system 100 to better adapt limited 10 processing resources as afunction of application predictions generated by a machine learningsystem, thereby providing better overall IO processing performanceacross multiple distinct applications.

It should be noted that paths may be added or deleted between the hostdevices 102 and the storage array 105 in the system 100. For example,the addition of one or more new paths from host device 102-1 to thestorage array 105 or the deletion of one or more existing paths from thehost device 102-1 to the storage array 105 may result from respectiveaddition or deletion of at least a portion of the storage devices 106 ofthe storage array 105.

Addition or deletion of paths can also occur as a result of zoning andmasking changes or other types of storage system reconfigurationsperformed by a storage administrator or other user.

In some embodiments, paths are added or deleted in conjunction withaddition of a new storage array or deletion of an existing storage arrayfrom a storage system that includes multiple storage arrays, possibly inconjunction with configuration of the storage system for at least one ofa migration operation and a replication operation.

For example, a storage system may include first and second storagearrays, with data being migrated from the first storage array to thesecond storage array prior to removing the first storage array from thestorage system.

As another example, a storage system may include a production storagearray and a recovery storage array, with data being replicated from theproduction storage array to the recovery storage array so as to beavailable for data recovery in the event of a failure involving theproduction storage array.

In these and other situations, path discovery scans may be repeated asneeded in order to discover the addition of new paths or the deletion ofexisting paths.

A given path discovery scan can be performed utilizing knownfunctionality of conventional MPIO drivers, such as PowerPath® drivers.

The path discovery scan in some embodiments may be further configured toidentify one or more new LUNs or other logical storage volumesassociated with the one or more new paths identified in the pathdiscovery scan. The path discovery scan may comprise, for example, oneor more bus scans which are configured to discover the appearance of anynew LUNs that have been added to the storage array 105 as well todiscover the disappearance of any existing LUNs that have been deletedfrom the storage array 105.

The MPIO driver 112-1 in some embodiments comprises a user-space portionand a kernel-space portion. The kernel-space portion of the MPIO driver112-1 may be configured to detect one or more path changes of the typementioned above, and to instruct the user-space portion of the MPIOdriver 112-1 to run a path discovery scan responsive to the detectedpath changes. Other divisions of functionality between the user-spaceportion and the kernel-space portion of the MPIO driver 112-1 arepossible.

For each of one or more new paths identified in the path discovery scan,the host device 102-1 may be configured to execute a host registrationoperation for that path. The host registration operation for a given newpath illustratively provides notification to the storage array 105 thatthe host device 102-1 has discovered the new path.

As indicated previously, the host device 102-1 may be configured togenerate a notification for delivery to a host administrator responsiveto an application prediction. Additionally or alternatively, the hostdevice 102-1 may be configured to generate a notification for deliveryto the storage array 105 responsive to an application prediction. Othertypes of reporting arrangements are utilized in other embodiments.

The above-described functions of the MPIO driver 112-1 are carried outat least in part under the control of its path selection logic 114-1.For example, the path selection logic 114-1 is illustratively configuredto control performance of the steps of the process to be described belowin conjunction with FIG. 2 . In other embodiments, one or more suchsteps can be more generally performed by the host device 102-1. Othersystem components such as storage array 105 can also participate in theFIG. 2 process.

It is assumed that each of the other MPIO drivers 112 is configured in amanner similar to that described above and elsewhere herein for thefirst MPIO driver 112-1. The other host devices 102 of the system 100are therefore also configured to communicate over the SAN 104 with thestorage array 105 and the MPIO drivers 112 of such other host devicesare each similarly configured to deliver IO operations from itscorresponding one of the sets of IO queues 110 to the storage array 105over selected paths through the SAN 104, and to perform at leastportions of the disclosed functionality for application predictionthrough machine learning based analysis of IO patterns.

Accordingly, functionality described above in the context of the firstMPIO driver 112-1 and the first host device 102-1 is assumed to besimilarly performed by each of the other MPIO drivers 112-2 through112-N and/or more generally by their respective host devices 102-2through 102-N.

The MPIO drivers 112 may be otherwise configured utilizing well-knownMPIO functionality such as that described in K. Piepho, “Dell EMC SCSeries Storage: Microsoft Multipath I/O,” Dell EMC Engineering, June2017, which is incorporated by reference herein. Such conventional MPIOfunctionality is suitably modified in illustrative embodiments disclosedherein to support application prediction through machine learning basedanalysis of IO patterns.

Although in some embodiments certain commands used by the host devices102 to communicate with the storage array 105 illustratively compriseSCSI commands, other types of commands and command formats can be usedin other embodiments. For example, some embodiments can implement IOoperations utilizing command features and functionality associated withNVM Express (NVMe), as described in the NVMe Specification, Revision1.3, May 2017, which is incorporated by reference herein. Other storageprotocols of this type that may be utilized in illustrative embodimentsdisclosed herein include NVMe over Fabric, also referred to as NVMeoF.

The storage array 105 in the present embodiment is assumed to comprise apersistent memory that is implemented using a flash memory or other typeof non-volatile memory of the storage array 105. More particularexamples include NAND-based flash memory or other types of non-volatilememory such as resistive RAM, phase change memory, spin torque transfermagneto-resistive RAM (STT-MRAM) and Intel Optane™ devices based on 3DXPoint™ memory. The persistent memory is further assumed to be separatefrom the storage devices 106 of the storage array 105, although in otherembodiments the persistent memory may be implemented as a designatedportion or portions of one or more of the storage devices 106. Forexample, in some embodiments the storage devices 106 may compriseflash-based storage devices, as in embodiments involving all-flashstorage arrays.

As indicated previously, the storage array 105 in the present embodimentcomprises at least one machine learning based application predictor 120.The machine learning based application predictor 120 illustrativelycomprises at least a portion of what is more generally referred toherein as a “machine learning system.” The machine learning basedapplication predictor 120 is shown in dashed outline, as it represents acomponent that can be additionally or alternatively implemented in oneor more of the host devices 102.

Accordingly, in some embodiments, a machine learning system can comprisea plurality of machine learning based application predictors implementedin the storage array 105 and/or at least a subset of the host devices102. The machine learning system in other embodiments can be entirelyseparate from the host devices 102 and the storage array 105. Forexample, the machine learning system can be implemented in anotherprocessing platform of system 100 accessible to the host devices 102and/or the storage array 105 over the SAN 104 or another network notexplicitly shown in the figure.

The storage array 105 in the present embodiment further comprisesadditional components such as response time control module 122 and IOoperation priority queues 124, illustratively configured to make use ofthe above-described persistent memory. For example, the response timecontrol module 122 may be used to implement storage array basedadjustments in response time for particular IO operations based at leastin part on service level objective (SLO) information stored by thestorage array 105 in its persistent memory. The response time controlmodule 122 operates in conjunction with the IO operation priority queues124.

The storage array 105 utilizes its IO operation priority queues 124 toprovide different levels of performance for IO operations. For example,the IO operation priority queues 124 may have respective differentpriority levels. The storage array 105 may be configured to providedifferent priority levels for different ones of the IO operations byassigning different ones of the TO operations to different ones of theIO operation priority queues 124. The IO operation priority queues 124are illustratively associated with respective SLOs for processing of IOoperations in the storage array 105.

Process tags may be used in assigning different ones of the IOoperations to different ones of the IO operation priority queues 124, asdisclosed in U.S. patent application Ser. No. 15/849,828, filed Dec. 21,2017 and entitled “Storage System with Input-Output Performance ControlUtilizing Application Process Detection,” which is incorporated byreference herein.

As mentioned above, communications between the host devices 102 and thestorage array 105 may utilize PCIe connections or other types ofconnections implemented over one or more networks. For example,illustrative embodiments can use interfaces such as Serial Attached SCSI(SAS) and Serial ATA (SATA). Numerous other interfaces and associatedcommunication protocols can be used in other embodiments.

The storage array 105 in some embodiments may be implemented as part ofcloud infrastructure in the form of a cloud-based system such as anAmazon Web Services (AWS) system. Other examples of cloud-based systemsthat can be used to provide at least portions of the storage array 105and possibly other portions of system 100 include Google Cloud Platform(GCP) and Microsoft Azure.

The storage array 105 may additionally or alternatively be configured toimplement multiple distinct storage tiers of a multi-tier storagesystem. By way of example, a given multi-tier storage system maycomprise a fast tier or performance tier implemented using flash storagedevices, and a capacity tier implemented using hard disk drive devices.A wide variety of other types of server-based flash storage devices andmulti-tier storage systems can be used in other embodiments, as will beapparent to those skilled in the art. The particular storage devicesused in a given storage tier may be varied depending on the particularneeds of a given embodiment, and multiple distinct storage device typesmay be used within a single storage tier. As indicated previously, theterm “storage device” as used herein is intended to be broadlyconstrued, and so may encompass, for example, disk drives, flash drives,solid-state drives, hybrid drives or other types of storage products anddevices, or portions thereof, and illustratively include logical storagedevices such as LUNs.

As another example, the storage array 105 may be used to implement oneor more storage nodes in a cluster storage system comprising a pluralityof storage nodes interconnected by one or more networks.

It should therefore be apparent that the term “storage array” as usedherein is intended to be broadly construed, and may encompass multipledistinct instances of a commercially-available storage array.

For example, the storage array 105 may comprise one or more storagearrays such as VNX®, Symmetrix VMAX® and Unity™ storage arrays,commercially available from Dell EMC. Other types of storage productsthat can be used in implementing a given storage system in illustrativeembodiments include software-defined storage products such as ScaleIO™,cloud storage products such as Elastic Cloud Storage (ECS), object-basedstorage products such as Atmos, scale-out all-flash storage arrays suchas XtremIO™, and scale-out NAS clusters comprising Isilon® platformnodes and associated accelerators, all from Dell EMC. Combinations ofmultiple ones of these and other storage products can also be used inimplementing a given storage system in an illustrative embodiment.

These and other storage systems can be part of what is more generallyreferred to herein as a processing platform comprising one or moreprocessing devices each comprising a processor coupled to a memory. Agiven such processing device may correspond to one or more virtualmachines or other types of virtualization infrastructure such as Dockercontainers or other types of LXCs. As indicated above, communicationsbetween such elements of system 100 may take place over one or morenetworks.

The term “processing platform” as used herein is intended to be broadlyconstrued so as to encompass, by way of illustration and withoutlimitation, multiple sets of processing devices and associated storagesystems that are configured to communicate over one or more networks.For example, distributed implementations of the host devices 102 arepossible, in which certain ones of the host devices 102 reside in onedata center in a first geographic location while other ones of the hostdevices 102 reside in one or more other data centers in one or moreother geographic locations that are potentially remote from the firstgeographic location. Thus, it is possible in some implementations of thesystem 100 for different ones of the host devices 102 to reside indifferent data centers than the storage array 105.

Numerous other distributed implementations of the host devices 102and/or the storage array 105 are possible. Accordingly, the storagearray 105 can also be implemented in a distributed manner acrossmultiple data centers.

It is to be appreciated that these and other features of illustrativeembodiments are presented by way of example only, and should not beconstrued as limiting in any way. Accordingly, different numbers, typesand arrangements of system components such as host devices 102, SAN 104,storage array 105, storage devices 106, sets of IO queues 110, MPIOdrivers 112, path selection logic 114 and machine learning basedapplication predictor 120 can be used in other embodiments.

It should also be understood that the particular sets of modules andother components implemented in the system 100 as illustrated in FIG. 1are presented by way of example only. In other embodiments, only subsetsof these components, or additional or alternative sets of components,may be used, and such components may exhibit alternative functionalityand configurations.

For example, in other embodiments, the application prediction techniquesdisclosed herein can be utilized to recognize, identify or otherwisepredict applications that are running in a storage system rather than onone or more host devices. In such an embodiment, the storage system canautomatically adjust its IO processing resource allocations based atleast in part on the application predictions generated by a machinelearning system, possibly implemented within the storage system itself.For example, the storage system can set different priorities fordifferent application types.

The operation of the information processing system 100 will now bedescribed in further detail with reference to the flow diagram of theillustrative embodiment of FIG. 2 . The process as shown includes steps200 through 206, and is suitable for use in the system 100 but is moregenerally applicable to other types of systems comprising at least onehost device and a storage system. The storage system in this embodimentis assumed to comprise at least one storage array having a plurality ofstorage devices. The storage devices can include logical storage devicessuch as LUNs or other logical storage volumes.

The steps of the FIG. 2 process are illustratively performed at least inpart by or under the control of an MPIO driver of a given host device.For example, portions of the process may be carried out under thecontrol of software, firmware and/or hardware of an instance of pathselection logic deployed within the MPIO driver. Other arrangements ofhost device components and/or other system components can be configuredto perform at least portions of one or more of the steps in otherembodiments.

In step 200, an MPIO driver collects information characterizing IOpatterns for respective ones of multiple applications executing on ahost device.

The IO pattern for a given one of the applications illustrativelycomprises a pattern of IO operations generated by the application anddelivered from the host device to the storage system over selected pathsunder the control of an MPIO driver. For example, one or more IOoperations selected from one or more queues of a given set of queues ofthe host device are illustratively directed to a particular logicalstorage device of the storage array over a selected one of a pluralityof available paths, with each such path being associated with adifferent initiator-target pair, where the initiators compriserespective HBAs of the host device and the targets comprise respectiveports of the storage array.

At least some of the paths from the host device to the storage array maybe determined using an initial path discovery scan performed inconjunction with booting of the host device. It is assumed for thedescription of this embodiment that the host device has discovered aplurality of paths to the storage array, and further that at least onelogical storage device of the storage array is visible to the hostdevice on each of the paths.

The path discovery scan can be repeated responsive to one or moredetected path changes or under other specified conditions. For example,a storage administrator or other user may perform zoning and/or maskingchanges to the storage array that result in at least one new pathbecoming available to the host device. Accordingly, the set of multiplepaths over which IO operations are delivered from the host device toparticular logical devices of the storage array can change over time.

In step 202, the MPIO driver provides the collected information to amachine learning system. The machine learning system illustrativelycomprises at least one machine learning based application predictor, andcan be implemented at least in part in the MPIO driver, itscorresponding host device and/or the storage array. For example, adistributed machine learning system can be used in some embodiments,with multiple machine learning based application predictors implementedon respective ones of multiple host devices and/or the storage array.Alternatively, the machine learning system in other embodiments can beimplemented on a separate processing platform comprising at least oneprocessing device configured to communicate with the host device and/orthe storage array over a network. The term “machine learning system” asused herein is therefore intended to be broadly construed, so as toencompass these and other possible implementations.

In step 204, a determination is made as to whether or not the trainingand testing of a machine learning model utilized by the machine learningsystem to recognize the IO patterns of different applications has beencompleted. If the training and testing of the machine learning model isnot yet complete, the process returns to step 200 as indicated in orderto collect additional information characterizing IO patterns for use inthe training and testing of the machine learning model. Otherwise, thetraining and testing of the machine learning model is complete, and theprocess moves to step 206.

In step 206, the machine learning system utilizes the collectedinformation to generate one or more application predictions based atleast in part on analysis of observed IO patterns.

As indicated in the figure, one or more initial iterations of steps 200and 202 provide collected information for training and testing of themachine learning model, and subsequent iterations provide collectedinformation for use by the trained and tested machine learning model ingenerating application predictions.

Additional or alternative steps can be used in other embodiments. Forexample, the MPIO driver or other components of the host device and/orstorage array can perform one or more other automated actions responsiveto application predictions. Examples of additional or alternativeautomated actions that may be performed responsive to applicationpredictions are described elsewhere herein.

Multiple additional instances of the FIG. 2 process may be performed inrespective additional host devices that share the storage array. Asindicated above, the machine learning system referred to in step 206 maybe implemented in one or more of the host devices, the storage arrayand/or an external processing platform.

The particular processing operations and other system functionalitydescribed in conjunction with the flow diagram of FIG. 2 are presentedby way of illustrative example only, and should not be construed aslimiting the scope of the disclosure in any way. Alternative embodimentscan use other types of processing operations involving host devices,storage systems and functionality for application prediction throughmachine learning based analysis of IO patterns. For example, theordering of the process steps may be varied in other embodiments, orcertain steps may be performed at least in part concurrently with oneanother rather than serially. Also, one or more of the process steps maybe repeated periodically, or multiple instances of the process can beperformed in parallel with one another in order to implement a pluralityof different application prediction arrangements within different hostdevices and/or storage arrays of a given information processing system.

Functionality such as that described in conjunction with the flowdiagram of FIG. 2 can be implemented at least in part in the form of oneor more software programs stored in memory and executed by a processorof a processing device such as a computer or server. As will bedescribed below, a memory or other storage device having executableprogram code of one or more software programs embodied therein is anexample of what is more generally referred to herein as a“processor-readable storage medium.”

Referring now to FIG. 3 , another illustrative embodiment is shown. Inthis embodiment, an information processing system 300 comprising atleast one instance of path selection logic 314 is configured inaccordance with a layered system architecture that illustrativelyincludes a host device processor layer 330, an MPIO layer 332, an HBAlayer 334, a switch fabric layer 336, a storage array port layer 338 anda storage array processor layer 340. The host device processor layer330, the MPIO layer 332 and the HBA layer 334 are associated with one ormore host devices, the switch fabric layer 336 is associated with one ormore SANs or other types of networks, and the storage array port layer338 and storage array processor layer 340 are associated with one ormore storage arrays (“SAs”).

The MPIO layer 332 is an example of what is also referred to herein as amulti-path layer, and comprises one or more MPIO drivers implemented inrespective host devices. Each such MPIO driver illustratively comprisesan instance of path selection logic 314 configured substantially aspreviously described. Additional or alternative layers and pathselection logic arrangements can be used in other embodiments.

In the system 300, path selection logic 314 is configured to selectdifferent paths for sending JO operations from a given host device to astorage array. These paths as illustrated in the figure include a firstpath from a particular HBA denoted HBA1 through a particular switchfabric denoted SF1 to a particular storage array port denoted PORT1, anda second path from another particular HBA denoted HBA2 through anotherparticular switch fabric denoted SF2 to another particular storage arrayport denoted PORT2.

These two particular paths are shown by way of illustrative exampleonly, and in many practical implementations there will typically be amuch larger number of paths between the one or more host devices and theone or more storage arrays, depending upon the specific systemconfiguration and its deployed numbers of HBAs, switch fabrics andstorage array ports. For example, each host device in the FIG. 3embodiment can illustratively have a set of n paths to a shared storagearray, or alternatively different ones of the host devices can havedifferent numbers and types of paths to the storage array.

The path selection logic 314 of the MPIO layer 332 in this embodimentselects paths for delivery of IO operations to the one or more storagearrays having the storage array ports of the storage array port layer338.

The system 300 in this embodiment further implements a machine learningsystem comprising one or more application predictors 320A and/or one ormore application predictors 320B. The one or more application predictors320A are implemented in one or more host device processors of the hostdevice processor layer 330. The one or more application predictors 320Bare implemented in one or more storage array processors of the storagearray processor layer 340. Each of the application predictors 320comprises a machine learning based application predictor that is part ofa machine learning system configured to generate application predictionsbased at least in part on analysis of observed JO patterns as disclosedherein. A machine learning system in some embodiments comprises aprocessor-based system that includes at least one machine learning basedapplication predictor, and that utilizes a neural network or other typeof machine learning model to recognize particular applications executingon the host devices by analysis of observed IO patterns.

It is assumed in this embodiment that the MPIO layer 332 providesfunctionality associated with application prediction, under the controlof the path selection logic 314, possibly with involvement of other hostdevice components. For example, one or more MPIO drivers of the MPIOlayer 332 are each illustratively configured to collect informationcharacterizing IO patterns for each of a plurality of differentapplications executing on the host device processor layer 330, and toprovide the collected information to a machine learning systemcomprising one or more application predictors 320A and/or one or moreapplication predictors 320B. The path selection logic 314 isillustratively configured to collect such information and to provide itto at least one of the application predictors 320A and 320B.

Although the one or more application predictors 320A are shown in FIG. 3as being implemented within the host device processor layer 330, this isby way of illustrative example only. In other embodiments, the one ormore application predictors 320A can be implemented at least in part inthe MPIO layer 332. For example, a given MPIO driver of the MPIO layer332 can implement an application predictor in some embodiments. Numerousother arrangements of the application predictors 320A and 320B can beused. Like the machine learning based application predictor 120 of FIG.1 , the application predictors 320A and 320B are shown in dashedoutline, to indicate that their placement within the system 300 can varydepending upon the particular implementation of the machine learningsystem in a given embodiment.

Some implementations of the system 300 can include a relatively largenumber of host devices (e.g., 1000 or more host devices), although asindicated previously different numbers of host devices, and possiblyonly a single host device, may be present in other embodiments. Each ofthe host devices is typically allocated with a sufficient number of HBAsto accommodate predicted performance needs. In some cases, the number ofHBAs per host device is on the order of 4, 8 or 16 HBAs, although othernumbers of HBAs could be allocated to each host device depending uponthe predicted performance needs. A typical storage array may include onthe order of 128 ports, although again other numbers can be used basedon the particular needs of the implementation. The number of hostdevices per storage array port in some cases can be on the order of 10host devices per port. The HBAs of the host devices are assumed to bezoned and masked to the storage array ports in accordance with thepredicted performance needs, including user load predictions.

A given host device of system 300 can be configured to initiate anautomated path discovery process to discover new paths responsive toupdated zoning and masking or other types of storage systemreconfigurations performed by a storage administrator or other user. Forcertain types of host devices, such as host devices using particularoperating systems such as Windows, ESX or Linux, automated pathdiscovery via the MPIO drivers of a multi-path layer is typicallysupported. Other types of host devices using other operating systemssuch as AIX in some implementations do not necessarily support suchautomated path discovery, in which case alternative techniques can beused to discover paths. For example, the storage array in someembodiments can be configured to increment a counter if zoning andmasking has changed, and the MPIO driver of the host device can utilizea log sense command, a mode sense command or a “vendor unique” or VUcommand to determine the current counter value and trigger pathdiscovery based on a change in that value.

Another illustrative embodiment of application prediction throughmachine learning based analysis of IO patterns will now be describedwith reference to FIG. 4 . This figure includes three differentportions, denoted FIG. 4A, FIG. 4B and FIG. 4C, each showing a differentaspect of an example implementation of application prediction throughmachine learning based analysis of IO patterns.

FIG. 4A illustrates an example machine learning model in the presentembodiment. The machine learning model comprises a deep learning neuralnetwork model comprising an input layer, multiple hidden layers and anoutput layer. The hidden layers more particularly include two denselayers of 64 hidden units each, and two dropout layers also of 64 hiddenunits each, as listed in the table at the lower portion of the figure.The model is a sequential model, with a total of six layers including aninput layer with 9 units, the four hidden layers mentioned previously,and an output layer with five units. There are a total of 5,125trainable parameters in this particular machine learning model. Themodel is illustratively implemented using Google Colaboratory, Keras andTensorFlow.

Referring now to FIG. 4B, the upper portion of the figure illustratestraining of the model on a training set over 10 epochs, and the lowerportion of the figure determines the loss and accuracy of the trainedmodel on a testing set (“test set”). The training set in this examplecomprises 36,708 samples, and the test set comprises 9,177 samples. Theaccuracy of the trained model on the test set was found to be 99.84%.

FIG. 4C illustrates generation of an application prediction based on newdata characterizing an IO pattern. The prediction output in this exampleindicates that the predicted workload type determined by the trainedmodel based on the IO pattern of the new data is an Exchangeapplication, as opposed to some other type of application, such as anOracle application, a Sybase application, and/or a data warehousingapplication, that the model has been trained to recognize, identify orotherwise predict from a corresponding IO pattern.

The particular application prediction arrangements described above inconjunction with FIGS. 2, 3 and 4 are presented by way of illustrativeexample only. Numerous alternative arrangements of these and otherfeatures can be used in implementing other illustrative embodiments. Forexample, a wide variety of different machine learning models can be usedin other embodiments.

The illustrative embodiments disclosed herein can provide a number ofsignificant advantages relative to conventional arrangements.

For example, some embodiments configure a multi-path layer of one ormore host devices to include functionality for application predictionthrough machine learning based analysis of IO patterns.

In some embodiments, deep learning neural networks or other types ofneural networks, or more generally machine learning models, are utilizedto provide highly accurate application predictions.

Techniques for application prediction through machine learning basedanalysis of IO patterns as disclosed herein can be implemented at leastin part in a host device and/or a storage system. For example, the hostdevice and/or storage system can be configured to implement a machinelearning system comprising one or more machine learning basedapplication predictors. In other embodiments, the machine learningsystem may be external to the host device and storage system.

Illustrative embodiments allow a host device and/or a storage system tobetter adapt limited IO processing resources as a function ofapplication predictions generated by a machine learning system, therebyproviding better overall IO processing performance across multipledistinct applications.

These and other embodiments avoid the need for explicit host-basedtagging of individual IO operations, and are broadly applicable to awide variety of different use cases and application types.

For example, some embodiments are particularly useful in cloudinfrastructure deployments in which it would otherwise be very difficultto determine which applications are running on which hosts and usingwhich logical storage volumes. Application predictions generated inthese and other embodiments can facilitate the optimization or otherimprovement of application performance by permitting allocation of IOprocessing resources based on particular application needs.

It is to be appreciated that the particular advantages described aboveare associated with particular illustrative embodiments and need not bepresent in other embodiments. Also, the particular types of informationprocessing system features and functionality as illustrated in thedrawings and described above are exemplary only, and numerous otherarrangements may be used in other embodiments.

It was noted above that portions of an information processing system asdisclosed herein may be implemented using one or more processingplatforms. Illustrative embodiments of such platforms will now bedescribed in greater detail. These and other processing platforms may beused to implement at least portions of other information processingsystems in other embodiments. A given such processing platform comprisesat least one processing device comprising a processor coupled to amemory.

One illustrative embodiment of a processing platform that may be used toimplement at least a portion of an information processing systemcomprises cloud infrastructure including virtual machines implementedusing a hypervisor that runs on physical infrastructure. The cloudinfrastructure further comprises sets of applications running onrespective ones of the virtual machines under the control of thehypervisor. It is also possible to use multiple hypervisors eachproviding a set of virtual machines using at least one underlyingphysical machine. Different sets of virtual machines provided by one ormore hypervisors may be utilized in configuring multiple instances ofvarious components of the system.

These and other types of cloud infrastructure can be used to providewhat is also referred to herein as a multi-tenant environment. One ormore system components such as virtual machines, or portions thereof,are illustratively implemented for use by tenants of such a multi-tenantenvironment.

Cloud infrastructure as disclosed herein can include cloud-based systemssuch as Amazon Web Services, Google Cloud Platform and Microsoft Azure.Virtual machines provided in such systems can be used to implement afast tier or other front-end tier of a multi-tier storage system inillustrative embodiments. A capacity tier or other back-end tier of sucha multi-tier storage system can be implemented using one or more objectstores such as Amazon S3, Google Cloud Platform Cloud Storage, andMicrosoft Azure Blob Storage.

The cloud infrastructure additionally or alternatively comprises aplurality of containers illustratively implemented using respectiveoperating system kernel control groups of one or more container hostdevices. For example, a given container of cloud infrastructureillustratively comprises a Docker container or other type of LXCimplemented using a kernel control group. The containers may run onvirtual machines in a multi-tenant environment, although otherarrangements are possible. The containers may be utilized to implement avariety of different types of functionality within the system 100. Forexample, containers can be used to implement respective compute nodes orstorage nodes of a cloud-based system. Again, containers may be used incombination with other virtualization infrastructure such as virtualmachines implemented using a hypervisor.

Another illustrative embodiment of a processing platform that may beused to implement at least a portion of an information processing systemcomprises a plurality of processing devices which communicate with oneanother over at least one network. The network may comprise any type ofnetwork, including by way of example a global computer network such asthe Internet, a WAN, a LAN, a satellite network, a telephone or cablenetwork, a cellular network, a wireless network such as a WiFi or WiMAXnetwork, or various portions or combinations of these and other types ofnetworks.

Each processing device of the processing platform comprises a processorcoupled to a memory. The processor may comprise a microprocessor, amicrocontroller, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a graphics processing unit (GPU)or other type of processing circuitry, as well as portions orcombinations of such circuitry elements. The memory may comprise randomaccess memory (RAM), read-only memory (ROM), flash memory or other typesof memory, in any combination. The memory and other memories disclosedherein should be viewed as illustrative examples of what are moregenerally referred to as “processor-readable storage media” storingexecutable program code of one or more software programs.

Articles of manufacture comprising such processor-readable storage mediaare considered illustrative embodiments. A given such article ofmanufacture may comprise, for example, a storage array, a storage diskor an integrated circuit containing RAM, ROM, flash memory or otherelectronic memory, or any of a wide variety of other types of computerprogram products. The term “article of manufacture” as used hereinshould be understood to exclude transitory, propagating signals.

Also included in the processing device is network interface circuitry,which is used to interface the processing device with the network andother system components, and may comprise conventional transceivers.

As another example, portions of a given processing platform in someembodiments can comprise converged infrastructure such as VxRail™,VxRack™, VxRack™ FLEX, VxBlock™ or Vblock® converged infrastructure fromDell EMC.

Again, these particular processing platforms are presented by way ofexample only, and other embodiments may include additional oralternative processing platforms, as well as numerous distinctprocessing platforms in any combination, with each such platformcomprising one or more computers, servers, storage devices or otherprocessing devices.

It should therefore be understood that in other embodiments differentarrangements of additional or alternative elements may be used. At leasta subset of these elements may be collectively implemented on a commonprocessing platform, or each such element may be implemented on aseparate processing platform.

Also, numerous other arrangements of computers, servers, storage devicesor other components are possible in an information processing system asdisclosed herein. Such components can communicate with other elements ofthe information processing system over any type of network or othercommunication media.

As indicated previously, components of an information processing systemas disclosed herein can be implemented at least in part in the form ofone or more software programs stored in memory and executed by aprocessor of a processing device. For example, at least portions of thefunctionality of host devices 102, SAN 104 and storage array 105 areillustratively implemented in the form of software running on one ormore processing devices. As a more particular example, the instances ofpath selection logic 114 may be implemented at least in part insoftware, as indicated previously herein.

It should again be emphasized that the above-described embodiments arepresented for purposes of illustration only. Many variations and otheralternative embodiments may be used. For example, the disclosedtechniques are applicable to a wide variety of other types ofinformation processing systems, utilizing other arrangements of hostdevices, networks, storage systems, storage arrays, storage devices,processors, memories, IO queues, MPIO drivers, path selection logic,application predictors, machine learning systems, machine learningmodels and additional or alternative components. Also, the particularconfigurations of system and device elements and associated processingoperations illustratively shown in the drawings can be varied in otherembodiments. For example, a wide variety of different MPIO driverconfigurations and associated application prediction arrangements can beused in other embodiments. Moreover, the various assumptions made abovein the course of describing the illustrative embodiments should also beviewed as exemplary rather than as requirements or limitations. Numerousother alternative embodiments within the scope of the appended claimswill be readily apparent to those skilled in the art.

What is claimed is:
 1. An apparatus comprising: a host device configuredto communicate over a network with a storage system comprising aplurality of storage devices; the host device comprising a multi-pathinput-output driver configured to control delivery of input-outputoperations from the host device to the storage system over selected onesof a plurality of paths through the network; wherein the multi-pathinput-output driver is further configured: to collect informationcharacterizing input-output patterns for each of a plurality ofdifferent applications executing on the host device; and to provide thecollected information to a machine learning system; wherein the machinelearning system utilizes the collected information to generate one ormore application predictions based at least in part on analysis ofobserved input-output patterns; wherein the machine learning systemgenerates a first application prediction indicating a first applicationtype responsive to detection of a first input-output pattern, andgenerates a second application prediction indicating a secondapplication type different than the first application type responsive todetection of a second input-output pattern different than the firstinput-output pattern; wherein collecting information characterizinginput-output patterns for each of a plurality of different applicationsexecuting on the host device comprises determining, based at least inpart on one or more counts of read requests, write requests or othertypes of input-output operations, statistical distributions of numbersof input-output operations having payload sizes falling within specifiedranges over a given time period; and wherein the one or more applicationpredictions generated by the machine learning system are utilized tomodify a path selection algorithm implemented by the multi-pathinput-output driver for selecting paths for delivery of input-outputoperations from the host device to the storage system through thenetwork.
 2. The apparatus of claim 1 wherein the machine learning systemis implemented at least in part in the host device.
 3. The apparatus ofclaim 2 wherein the machine learning system is implemented at least inpart in the multi-path input-output driver of the host device.
 4. Theapparatus of claim 1 wherein the machine learning system is implementedat least in part in the storage system.
 5. The apparatus of claim 1wherein collecting information characterizing input-output patterns foreach of a plurality of different applications executing on the hostdevice comprises collecting information for use in development of amachine learning model for deployment in the machine learning system. 6.The apparatus of claim 5 wherein the collected information for use indevelopment of the machine learning model comprises one or more of: atraining set for training the machine learning model to recognizeparticular types of applications from their respective input-outputpatterns; and a testing set for evaluating performance of the machinelearning model in recognizing particular types of applications fromtheir respective input-output patterns.
 7. The apparatus of claim 1wherein collecting information characterizing input-output patterns foreach of a plurality of different applications executing on the hostdevice comprises collecting information for use by a deployed machinelearning model of the machine learning system in generating the one ormore application predictions.
 8. The apparatus of claim 1 wherein themulti-path input-output driver is further configured: to receive the oneor more application predictions generated by the machine learningsystem.
 9. The apparatus of claim 1 wherein a given applicationprediction generated by the machine learning system comprisesinformation identifying a particular one of a plurality of possibleapplication types.
 10. The apparatus of claim 1 wherein at least one ofthe host device and the storage system is configured to utilize the oneor more application predictions generated by the machine learning systemto adjust one or more parameters related to its processing ofinput-output operations for the corresponding applications.
 11. Theapparatus of claim 10 wherein at least one of the host device and thestorage system is further configured: to allocate a relatively highlevel of processing resources to input-output operations associated witha first application prediction; and to allocate a relatively low levelof processing resources to input-output operations associated with asecond application prediction different than the first applicationprediction.
 12. The apparatus of claim 1 further comprising one or moreadditional host devices each configured to communicate over the networkwith the storage system and wherein each additional host devicecomprises a multi-path input-output driver configured to controldelivery of input-output operations from that host device to the storagesystem over selected ones of a plurality of paths through the network.13. The apparatus of claim 1 wherein the paths are associated withrespective initiator-target pairs wherein each of the initiatorscomprises a corresponding one of a plurality of host bus adaptors of thehost device and each of the targets comprises a corresponding one of aplurality of ports of the storage system.
 14. A method comprising:controlling, in a multi-path input-output driver of a host device,delivery of input-output operations from the host device to a storagesystem over selected ones of a plurality of paths through a network;collecting, in the multi-path input-output driver, informationcharacterizing input-output patterns for each of a plurality ofdifferent applications executing on the host device; and providing thecollected information from the multi-path input-output driver to amachine learning system; wherein the machine learning system utilizesthe collected information to generate one or more applicationpredictions based at least in part on analysis of observed input-outputpatterns; wherein the machine learning system generates a firstapplication prediction indicating a first application type responsive todetection of a first input-output pattern, and generates a secondapplication prediction indicating a second application type differentthan the first application type responsive to detection of a secondinput-output pattern different than the first input-output pattern;wherein collecting information characterizing input-output patterns foreach of a plurality of different applications executing on the hostdevice comprises determining, based at least in part on one or morecounts of read requests, write requests or other types of input-outputoperations, statistical distributions of numbers of input-outputoperations having payload sizes falling within specified ranges over agiven time period; and wherein the one or more application predictionsgenerated by the machine learning system are utilized to modify a pathselection algorithm implemented by the multi-path input-output driverfor selecting paths for delivery of input-output operations from thehost device to the storage system through the network.
 15. The method ofclaim 14 wherein the multi-path input-output driver is furtherconfigured: to receive the one or more application predictions generatedby the machine learning system.
 16. The method of claim 14 wherein atleast one of the host device and the storage system utilizes the one ormore application predictions generated by the machine learning system toadjust one or more parameters related to its processing of input-outputoperations for the corresponding applications.
 17. A computer programproduct comprising a non-transitory processor-readable storage mediumhaving stored therein program code of one or more software programs,wherein the program code, when executed by a host device comprising amulti-path input-output driver, the host device being configured tocommunicate over a network with a storage system, causes the hostdevice: to control, in the multi-path input-output driver of the hostdevice, delivery of input-output operations from the host device to thestorage system over selected ones of a plurality of paths through thenetwork; to collect, in the multi-path input-output driver, informationcharacterizing input-output patterns for each of a plurality ofdifferent applications executing on the host device; and to provide thecollected information from the multi-path input-output driver to amachine learning system; wherein the machine learning system utilizesthe collected information to generate one or more applicationpredictions based at least in part on analysis of observed input-outputpatterns; wherein the machine learning system generates a firstapplication prediction indicating a first application type responsive todetection of a first input-output pattern, and generates a secondapplication prediction indicating a second application type differentthan the first application type responsive to detection of a secondinput-output pattern different than the first input-output pattern;wherein collecting information characterizing input-output patterns foreach of a plurality of different applications executing on the hostdevice comprises determining, based at least in part on one or morecounts of read requests, write requests or other types of input-outputoperations, statistical distributions of numbers of input-outputoperations having payload sizes falling within specified ranges over agiven time period; and wherein the one or more application predictionsgenerated by the machine learning system are utilized to modify a pathselection algorithm implemented by the multi-path input-output driverfor selecting paths for delivery of input-output operations from thehost device to the storage system through the network.
 18. The computerprogram product of claim 17 wherein the multi-path input-output driveris further configured: to receive the one or more applicationpredictions generated by the machine learning system.
 19. The computerprogram product of claim 17 wherein at least one of the host device andthe storage system utilizes the one or more application predictionsgenerated by the machine learning system to adjust one or moreparameters related to its processing of input-output operations for thecorresponding applications.
 20. The computer program product of claim 17wherein a given application prediction generated by the machine learningsystem comprises information identifying a particular one of a pluralityof possible application types.